Search for: All records

In this paper, we investigate the threat of drones equipped with recording devices, which capture videos of individuals typing on their mobile devices and extract the touch input such as passcodes from the videos. Deploying this kind of attack from the air is significantly challenging because of camera vibration and movement caused by drone dynamics and the wind. Our algorithms can estimate the motion trajectory of the touching finger, and derive the typing pattern and then touch inputs. Our experiments show that we can achieve a high success rate against both tablets and smartphones with a DJI Phantom drone from a long distance. A 2.5" NEUTRON mini drone flies outside a window and also achieves a high success rate against tablets behind the window. To the best of our knowledge, we are the first to systematically study drones revealing user inputs on mobile devices and use the finger motion trajectory alone to recover passcodes typed on mobile devices.